Write a custom CUDA kernel to optimize the Flatten-T Swish (FTSwish) activation function.

The mathematical definition is:
f(x) = (x * sigmoid(x) + T) if x >= 0
f(x) = T                    if x < 0
where T is a scalar threshold parameter.

Problem Analysis:
The standard PyTorch implementation using torch.where(condition, a, b) is inefficient for this specific function.
1. Redundant Computation: torch.where evaluates both branches for all elements. This means expensive exponential instructions (for sigmoid) are executed even for negative inputs where the result is simply T.
2. Memory Bandwidth: The operation generates intermediate tensors (masks and sigmoid results), increasing global memory traffic.

Optimization Strategy: Fused Conditional Kernel with Vectorized Access

1. Fused Logic with Early Exit: Create a single CUDA kernel. Inside the kernel, check the sign of the input x. If x < 0, immediately write T. Only if x >= 0, compute the sigmoid and the full expression. This branching saves significant compute resources.

2. Vectorized Memory Access: Use float4 data types to load and store 128 bits (4 floats) per instruction. This maximizes global memory bandwidth utilization, which is critical for element-wise operations.

3. Fast Math Intrinsics: Use hardware-accelerated intrinsics like __expf inside the sigmoid calculation to reduce arithmetic latency.

4. Grid-Stride Loop: Implement the kernel using a grid-stride loop to handle arbitrary tensor sizes robustly.Write a custom CUDA kernel to optimize the Flatten-T Swish (FTSwish) activation function.

The mathematical definition is:
f(x) = (x * sigmoid(x) + T) if x >= 0
f(x) = T                    if x < 0
where T is a scalar threshold parameter.

Problem Analysis:
The standard PyTorch implementation using torch.where(condition, a, b) is inefficient for this specific function.
1. Redundant Computation: torch.where evaluates both branches for all elements. This means expensive exponential instructions (for sigmoid) are executed even for negative inputs where the result is simply T.
2. Memory Bandwidth: The operation generates intermediate tensors (masks and sigmoid results), increasing global memory traffic.

Optimization Strategy: Fused Conditional Kernel with Vectorized Access

1. Fused Logic with Early Exit: Create a single CUDA kernel. Inside the kernel, check the sign of the input x. If x < 0, immediately write T. Only if x >= 0, compute the sigmoid and the full expression. This branching saves significant compute resources.

2. Vectorized Memory Access: Use float4 data types to load and store 128 bits (4 floats) per instruction. This maximizes global memory bandwidth utilization, which is critical for element-wise operations.

3. Fast Math Intrinsics: Use hardware-accelerated intrinsics like __expf inside the sigmoid calculation to reduce arithmetic latency.

4. Grid-Stride Loop: Implement the kernel using a grid-stride loop to handle arbitrary tensor sizes robustly.
  
Here's an example to show you the syntax of inline embedding custom CUDA operators in torch: The example given architecture is:   
  
```python
import torch
import torch.nn as nn
import torch.nn.functional as F

BATCH_SIZE = 4096
DIM = 4096
SHAPE = (BATCH_SIZE, DIM)
T_VALUE = -0.2  # 阈值参数 T

class FlattenTSwish(nn.Module):
    """
    Flatten-T Swish Activation Function.
    Definition:
        f(x) = x * sigmoid(x) + T   if x >= 0
        f(x) = T                    if x < 0
    """
    def __init__(self, T=0.0):
        super(FlattenTSwish, self).__init__()
        self.T = T

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        # --- 优化的 PyTorch 实现 ---
        # 优化思路:
        # 利用数学等价性消除分支和显存拷贝。
        # 1. F.relu(x): 将负数变为 0。
        # 2. F.silu(...): 计算 x * sigmoid(x)。
        # 3. + T: 加上偏置。
        # 避免了 torch.where 的全量计算浪费和中间显存开销。
        return F.silu(F.relu(x)) + self.T

class Model(nn.Module):
    def __init__(self, T=0.0):
        super(Model, self).__init__()
        self.act = FlattenTSwish(T=T)
    
    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.act(x)

def get_inputs():
    x = torch.randn(SHAPE, dtype=torch.float32)
    return [x.contiguous()]

def get_init_inputs():
    return [T_VALUE]